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ABSTRACT 

The Joreskog and Sorbom LISREL (linear 
relations) method is investigated as aji alternative to 
analysis.;in studies of aptitude- treatment interactions 
S'olve problems caused by unreliability of measurements 
sets of W^iables. A study reparted by H.J. Behr is rea 
study investigated relations between verbal and figural 
variables and outcome variables within one verbal and o 
treatment^ teaching modulus seven arithmetic to prospec 
elementary tea<|hers. The relations between latent aptit 
and' latent outcome variables are studied in four LISREL 
€ither one- or ^two latent aptitude variables (interpreta 
genefal factor and verbal and figural ability^ respecti 
with' two sets of latent otitcome variables (interpretabl 
learning /retention' and computation speed/understanding) 
significant interaction is founds but tendencies toward 
are noted. It is concluded that LISREL has several adva 
analysis of ATI studies, but also that the power of the 
effects is low, particularly when, there is a high corre 
the latent aptitude variables. (Author/GDC) 
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ABSTRACT 



The Joreskog -and Sorbom LISREL method is .investigated a 
an alternative to regression^ an^alysis in studies of > • 
aptitude-treatment interactions (ATI) , to solve problem 
caused by unreliability of measurements arid by large 
' sets of variables. A study reported by M.J. Behr is 
reanalyzed. TJie study investigated relations' between 

V verbal and f igural aptitufle variables and outcome 

■ • ■ ') 

^^-^ variables within one verbal and one f igural treatment 
' teaching modulus seven^^^rithmetic . The; relations 
between latent aptitude variables and latent outcome 
variables are studied in 4 LISREL models with either 
1 or 2 latent aptitude variables, interpretable as a 
general factor and verbal and figural ability, respecti- 
vely, and with 2, sets of latent outcome variables, 
interpretable' as learning/retention and computational 
^peed/understanding. No significant interaction is 
found, but tendencies towards interaction are noted. 
It is concluded that LISHEL has several advantages in 
V the analysis 'of ATI studies, but also that the power " 
of the test^ of ATI effects is low, particularly when 

is'\a high correlation between the latent aptitude 
'^rariabl'es . ^ 



INTRODUCTION 



In research, on aptitude-treatment interactions (ATI) the 
interest is focus sed on joint effects of instructional 
treatment and individual differences. The standard procedure 
for anaj.yzing >data from ATI studies is to regress outcome 
variables one ^t a time on one or. more aptitude variables 
within treatment groups and to test 'for homogeneity/of 
the within treatment regression slopes (Cronbach & Snow, 
1977, ch. 3)-r I ' . 



However, the <6rdinary regression analysis of ATI data is 
not f ree^ from problems. The regression on an observed 
aptitude Variable is differen"^ from the 3?fegression, on 
the true , aptitude variable*^ when it is riot perfectly 
reliable; errors of measurement in the aptitude variables 
thus bias the tests and descriptions of ATI effects. 
. Cronbach and. Snow (1977) stated that "Ideally, every ATI 
^"^^^^udy would pxamine the regression of outcome onto the 
true aptitude score... This.., regression will have a 
different slope than the observed-score regression, and 
under some circumstances the apparent interaction may 
be radically altered. Important as this matter is, it 
has been ignored in ATI research to date..." (p. 34). 

■ 1 ■ ■ i 

The problenQi is severlje enough when only one aptitude vari^ 
able is^ considered (hut it is aggravated in multiple 
regression (Cronbach & Snow, 1977, p. 36). Furthermore, 
an additional complication arises^^n the regression based 
"approach to the analysis of ATI (data when there are many 
variables. Then a great many regression coefficients are 
estimated and tested, which makes for chance significan- 
cies and tends to. give rise to complex patterns of results 
which are' hard to interpret. 

In this paper an alternative approacji to the analysis of 
ATI data will be illustrated the LISREL n^ethod of 



/ Joreskog and Sorbom (see Joreskog, 19 70, 19 73, 19 74; 
Joreskog & Sorbom, 1976, 1977, 1978). LISREL (Linear " 
Structural RELations) is a method, and a computer pro- 
gram, for analysis of linear structural relations between ; 
•variables, .which may be either observed or latent. When 
latent variables are studied a factorial structure is 
imposed, on the o^Dse'rved variables, which structure serves 
to identify the latent variables and allows estimation 
.of the error variances in the observed variables^ In 
this'way ^'^veral observed variables may be arediiced to^, 
^ fewer latent variables and relations between true 

variables rather t^han observed variables can be studied. 

It cannot 'be taken for granted , however , that LISREL only 
has advantages when- applied to ar^alysis o:^ATI studies: 
The method is' built on st^rong assumptions concerning the 
^ nature of the daj^a;^ not only consistency of th^ estimates 

is of importance but also their variance, just to mention ' 
.two possible sources of problems. ^There seems) T:hus to be a 
need for empirical studies of the applicability of this 
alterna'tive method in the analysis of ATI -clata. 

% - 

One pU'rpQse of the present stiidy is to ^make such an 
. appraisal df the problems ^^nd virtues of LISREL. The 
f ^ .^study is a reanalysis of a' "study presentra^ by Behr (1967), 

which was chosen because it included several aptitude 
and outcome variables and because 'the , present authors 
^yare familiar with the' substantiye problem studied by 
Behr. ^ > - ^ 



• ' Behr investigated the hypothesis that tests of verbal 
ab;Llity ar^ more h'lghly correlated with achievement in 
^» a verbal treatment than in a figural treatment and that 
^. -tests of figural ability are more highly correlated with 
achievement in a figural treatm^t than in a verbal 
'tr^atmeht (cf. Gustaf:^^on, 1976?, .This was^ one of the 
first ATI hypotheses to be suggested and seversal of the 
^ early 'ATI *sti^:es>. of which the Behr study^is one, investi- 

gated this hy^othesd'te . . - . n 

.... .r . -V \ . 



Behr used 14 aptitude variabels; about. half of them were 
verbal and half of t\\em were figural. One group of subjects 
studied a verbal-symbolic (VS) programmed teaching mater.ial 
and another group of sub jec ts studied a f igural-symbolic 
'(FS) teaching material. The study included 7 outcome 
variables'^'' (for a more detailed description of the Behr* 
study, see below)-. 4 

Behr investigated interactiqps by comparing the withih- 
treatinent regressions of .each of the dependent variables 
on each of the aptitude variables . About a dozen signi^ 
ficant interactions were found, and alnj^ost, all interactions 
were due to a steeper regression on verbal tests in the 

VS treatment than in /the FS , treatment . ^ 

, . ^ ' ' , \ ^ 

" ' ■' ■ * 

In their review of ATI research Cronbach and Snow (19 77, 
p. 286) made a simple reanalysis usii^v-f igures presented 
by Behr, For one outcome they' surftihie^l^g^' 3fegjression 
coefficients of adjT'^^he ^ figural lfe3^^;:^ejiara for 
the two treatments. This ^procedurqr .was"*ll^;^'ej^t^d for the 
verbal tests. The sum for both g^roilp^s of/^ tests j^as higher 
in the VS- treatmefit, even thduglis the'' dif f-erJenc'e^^iw^ 
smaller for the ^/ij^^al tests. Gror^ach .a^f^^^Snow. drew 
the conclusion that a general' factor was mpr^; associated 
with achievement in th^ VS ihan.' ih the FS trea'tmen-t . 

^ ' . ' '■■ ' - ■ 
Gustafsson (1974 , pp, 15-16 ;~^cf . Gustafsson, 1976, p-56) 

'*■ 

presente^'another simple reanalysis of the Behr study 
The most reliable tests were selected and one verbal 
and one figaral test at a time were, entered into separate 
multiple regression equations for the two treatments^^It 
was found that one figural test that had a highe^ zero- 
order' regression coefficient in the FS treatment, in 
{?he multipl-e .analysis came ouffwith a much higher partial 
regression coefficient in the FS treatment than ii^ the 
VS treatment. This result was interpreted as a veify 
weak .^tendency towards interaction wit^^ figural ability 



4 



and since the interaction with verbal ability seemed 
relatively well established it was concluded that the 
Behr study gave sotne support to the verbal/f igural ATI 
hypothesis. Three different conclusions have thus been 
drawn from the B6hr study: Behr himself stressed the 
results obtained with each particular aptitude variable? 
Cronbach and Snow interpreted the effects found as being , 
accountable for by general ability; and Gustafsson, 
finally, saw some value in the distinction between verbal 
and f igural abilities. In addition to the, methodological 
^purpose of studying the applicability of LISREL, another 
purpose will t>e to see which, if any, of these conclusions 
receives support. 



METHOD 



The Behr study 

The subject matter taught" in the Behr study was modulus seven 
arithmetic. In both the VS and FS treatments algebraic sionbols 
were used but in the VS treatment this information was supple- 
mented with verbal information. In the FS treatment f igural 
information was added to the symbolic presentation. Th6 f igural 
material was of low complexity, consisting for example of a^ 
circle with which the arithmetic operations were illustrated 
as movemen-ts along the circl^e. 

The, aptitude varibles were selected to correspond to cells 
in the Guilf ordM 1967 ) "Structure-of-Intellect (SI)" model. 
Of the tests, 6 had a f igural content (-F-) , 5 had a semantic 
(verbal) content (-M-) and 2 had a symbolic (numeric) (-S-) 
content. In addition there wa>s a test, called integration, 
which was not classifiable in the SI structure. This test 
qave verballv'formulated in^^Jto^ctions concerning directions. 
It can be hypothesized that this test measures figural 
ability? it comes quite close to the task which Brooks. 
(1967) in experimental studies has shown to be of a spatial/ 
figural kind. Two kinds of operations, cognition (C — ) and 
J 



memory (M — ), were represented among the aptitude variables. 
Most of tlte product categories were also represented among 
one or more of the tests: units ( — U) , classes ( — C) , 
relations ( — R) , transformations ( — T) and implications 
( — I) . Table 1 displays the tests used, their coding in 
the SI system, the number of ite^ns, ai^ the time limits. 



Insert Table 1 about here 



It can be observed that many of the tests are quite short, 
with a very limited testing time, which is partly due to 
the fact that Behr.in most cases used shortened versions 
of the tests. 

Three criterion measures were determined: Time used to 
study the program (TP) , a Learning Test (LT) score and a 
Retention T^st (RT) score. The LT was administered two days 
after the instruction and the RT two weeks after the 
instruction. 

The LT was written in five parts. Part- I was a speed test 
of modulus .seven addition and subtraction. Parts II, III,' 
IV and V all dealt with structural properties of the modulus 
seven system, but contained different kinds of items. Two 
sub-test scores were derived from the LT: one consistintf of 
the score on Part I (LA) and one consisting of the sums of 
scores on parts II-V (LB) . Behr used the two sub-test scores 
as dependent variables, along with the total LT score. 

The RT was a pajrallel form of LT and was written with other 
modulus" seven numbers in the items or with changes in the 
order of the questions or responses . Also from the RT two 

4 

sub-test scores (RA and RB) , corresponding to those in the 
LT, were derived and used as dependent variables along 
with the total RT score. . , , 

Subjects in the study were prospective elementary school 
.teachers, there being 120 and 108 subjects in the VS and 
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FS groups, respectively. The subjects were randomly assigned 
to treatments • 

Through assembling information presented by Behr in tables 
and appendices it was possible to construct the within 
treatment covariance matrices, liif must be mentioned, however, 
that one of the correlations presented by Behr was changed. 
For the FS group the correlation between CFR and MMU was 
reported to be .71. This correlation is by far the highest 
correlation for any two aptitude variables and is probably 
due to a typing error? it was therefore replaced with the 
value .17. ' ^ 

LISREL • 



:i^ 



The LISREL approach, and related ones, has been described 
in several publications (e.g. Joreskog, 1970, 1973,^1974; 
Joreskog & Sorbom, 1976, 1977 , 1978 ) Here only a very 
sketchy desci;ipti^n can be afforded. ^ . 

The LISREL model consists of two parts: the measurement 
models for the dependent and independent variables, in 
which latent variables (common factors) are defined in 
terms of the observed variables, and the linear structural 
equation model, in which the relations between the latent 
variables :are specified. 

There are two sets of observed variables y"= (72^/72 '^p^ 
and x"= (x^ , . . . ), corresponding to dependent and 
independent observed variables respectively, and two sets ^ 
of latent variables n"= ( r n2 ^ • • / n^^) and (^2.'^2' 

), corresponding to dependent and independent latent 
variables respectively. There are also vectors specifying 



the unique parts Q|^^e y and x variables, e"= (^i^^2' 



...,r ) and ffaiHPn^ , . . . , iS ) and a vector specifying the 
p ~ ' X z q 

residuals in. the structural equations system: r,"= {(.^^f^' 



In LISREL the relations between the latent variables and 
the observed variables, and the relations between the 
latent variables are specified in up to eigh't parameter 
matrices: ^ 



is a factor loading matrix of order q x for the- 

regression of the x variables on the ? variables. 

is a factor loading matrix of order p x m, for the 

regression of the y variables on^ the n variables. 

IS a diagoiial or symmetric matrix of order q x <j . - 

containg, the covariance matrix for the unique parts 

of the X variables. 
0^ is a diagonal or symmetric matrix of order p x p 

containg the covariance matrix for the unique parts 

of the y variables. 
$ is a diagon^f or symmetric matrix of order n x' n 

containg ^t}^(^—COVar^^^^ matrix of 'the ^ variables. 
^ is a diagonal or symmetric matrix of order m x m for 

the residuals (disturbance terms or errors in equations) . 
r is a coefficient matrix or order m x n for the structural 

relations between the E, and the n variables. 
6 is a coefficient matrix of order m x m for the structural 

relations among the n variables. 

The measurement models for the x -variables is written: 



~x^ 

and for the y variables it is written^ 



y = A n+G . 

The system of linear structural equations has the form: 

Specifying a LISREL model involves specification of the 
nature of each element ir^ the parameter matrices (the 
elements will be referred to. with small Gteek letters) . 



The elements can be of three kinds: a fixed parameter, 
i.e. the parameter is assigned a given value; a free 
pai:ameter, i,e. the parameter is to be estimated; and a 
constrained "parameter , i,e, the parameter is to be 
estimated but it is constrained to be equal to one or 
more other parameters. 

If the model is identified, if there is a unique 

estimate of each non-fixed parameter, the parameters can 
be estimated with maximum likelihood methods from the 
sample covariance matrices, using the Joreskog and Sor- 
bom ri9 78) LISREL4 program, for example. Each analysis 
of a fully identified model not only yields estimates of 
paramet^ers l?ut also an overall x test of the goodness 
of fit' to the model, ^along with standard errors of the 
estimated parameters. Through computing the differences 
between the values of the test statistics obtained with 
more and less constiTained models, i.e. models differing 
as to the number of parameters estimated, it is also 
possible to test the signigicance of subsets of {para- 
meters. 

So far the presentation of the LISREL model has only 
dealt with the case when there is one group only. However, 
the LISREL4 program handles several groups as well, and 
through cbnstraining parameters to be equal in several groups 
it is possible to test the equality between groups, either 
of all the estimated parameters or of subsets of the pa^ra- 
metersMcf . Sorbom, 19 78) . • 

In ATI applications of LISREL the main interest is of 
course centered oit the T coefficients, which roughly corres 
pond to the within-treatment regression coefficients and it 
is a rather straightf orwai*d process to estimate these 
within the treatment groups and to test them for equality 
between groups. 



RESULTS -i 

/ • .. . ' ' ^- ' 

*It^ was -suggested by JoreBkog' and Sorbom (1977) that it [ 

^ay be ^a good s'fcrategy -'to construct the .^full LISREL model 

"±n steps, .starting witfi the measurement models and then 
' ■ ■ ^ ' ' ' ■ , ■ ^ ^ .. 

only at a later ste^- fit these together through the linear 

-v- - >• • ' ^ - 

: stfcuctural^^quations . Tjp^i,S is the strategy followed here, 
-^aiid^ we. will start with the measur^nient model, for the 
'.aptxtudp variables. 

Measurement models- for the ap,titu^'e variables 

-^Each of the cells in the Guilford Sl-system supposedly 
defines a unique factor' and since in the Behr study only 
one tesyfe was sampled from each cell it could, from the 
SI point of view, be argued that no attempts should be 
made to account for these wi^h a smaller set of common 
factors. However, apart from the fact that the inter- 
correlations among the factors defined in the Sl-system 
have not been much studied, it could be argued that the 
Sl-system is so elaborate as tcT^e impractical. Cronbach 
and Snow said: ^ 

"if as many abilities as Guilford recognizes must 
be recognized, hypotheses about AT>I will have to 
be finely differentiated and very large samples 
will be needed to establish weightings for separate 
abilities. The prospects for sucessful ATI research 
would be much enhanced if it were decided that a system 
simpler than Guilford's accounts for the ability 
differences of long-run practical importance. " 
(Cronbach' & Snow, 1977, p. 155, emphasis in original). 

Cronbach ^and Snow (1977, pp. 155-160) made -a partial 
assessment of the SI system and presented some reanalyses 
of correlation matrices. They concluded, among other 
things, that tests with a similar. "content" tend to inter- 
correlate but that tests calling for the same kind of 
"product" not fi^ctionally similar. 



.There are thus strong , reasons ^to investigate the ^ 
possibilities to reduce- the many observed aptitude / 
• - variables in the Behr study to~ a _ smaller set ofi common 
factors. , -,--,-;r.-^ 

. When LISREL j.s applied to develop the measurement models 
a sequence of confirmatoiry factor analyses Is carriec^ 
out. To get some information about the dimensionality of' 
\ the. latent space onte -ordinary exploratory factor analysis 
was first performed within each treatment group. These. ^ 
analyses indicated, in thk first place, that. the ^wo -S- ' 
tests had very little in common with the other tests, and 
they were therefore .excluded . The sequence of eigenvalues 
indicated, secondly, that there in the groups were 3 or 4 
common factors. 'i;here were, howeverV some, differences in 
the size and pattern of loadings for the two groups. 

It seemed a good strategy to start with few factors in the 
confirmatory analyses, and then. add more factors if necessa- 
ry. A 1-f actor model was fitted, and also a 2-f actor model, 
with one factor defined by all the -M- tests, and the other 
. factor defined by the -F- tests. For each of these cases 
two solutions were made; one in which the parameters were 
specified to be the same in the two treatment groups and 
one in which no constraint of equality was imposed. The 
results from the tests of. goodness of fit. of these four, 
solutions have been entered in Table 2, along with the 
appropriate differences to test equality of the solutions 
in the groups and to test whether the 2-factor solution 
is significantly better than the 1-factor solution. _ / 



Insert Table 2 about here 



No signijficant difference is found between the solutions 
for the treatment groups. The te.sts of the differences 
between the 1- "and 2-factor solutions indicate a significantly 



\ 



11 



better^ fit for the 2-f actor solution. Furthemn^re, the 
overall evaluation of* fit of the 2-factor solution gives 
a value of the test statistic which is slightly lower 
than the critical value. Ttjus, even though the fit of 
the verb^l/f igural solution is far from perfect it is 
for the present purposes considered acceptable. 

Had higher standards^pf^^t been set it would of course 
have been natural tp' jjg^»I|3"-f actor , and perhaps a 4- 
factor model. Howevei^^WHCTi'. though it will, not be illustrated 
here w^ would like tp meBt.ion the- availability -of another 
strategy to obtain fjL't without adding further factors.* 
Often the specific parts of some tests correlate because 
the tests share certain characteristics, such as being 
administrered^ at a common occasion, or having similar content, 
or requiring the same type of answers and so on, In contrast 
with ordinary factor analysis the correlations between the 
errors of measurement need in LISREL rxot be /assumed to be 
zero, but can be set as free parameters to-be estimated 
(cf . Sorbom, 19 75) / which may at times be a better strategy 
than to introduce additional factors. 

In Table 3 the standardised factor loadings estimated 

within each treatment group are presented.. The factors 

will be labeled the figural and the verbal factor respectively. 



Insert Table 3 about here 



It is , however, difficult to characterize the nature of the 
factors more^ closely on the basis of the size of the loadings, 
since the differences between the groups tend to be almost 
as large as the differences between the tests. There is a 
tendency for the figural fa^ctor to display higher loadings 
in the' VS group than in the FS group. Tests of the equality 
of each loading show that among the figural tests there is 
one significant difference between the treatment groups; 



• \ ' • ^ 

■ ■ • . ■ ■ 1 

■ r ■ . ' 

«^ 

for MFU a t-value of 2,63 is obtained. The differences-^ are 
sinaller and less systematic for the verbal factor, but 
there is a significantdif'ference- for the MMR test (t= 1.97) • 

It can be observed that fot many of the tests the communa- , 
lity is very ^^ow\, One reason for this is that the reliabi- 
lity of the tests must' be^ low ^since they are often very 
short. Another reason might be that each test brings, much 
specif iqity J this is of ' course to be expected 'oh ;j^e basis 
of Sl-theary. With LISREL, however, it is." not in a model 
like this one possible to obtain separate estimates of ^the 
specificity and the errors of measurement as such. 

The factors are ver^ highly intercorrelatedj . in the VS 
group the correlation is .81 and in the FS group it is 
.80. Even though these correlations are significantly 
different from unity they are so high that there is very ^ 
little information in the verbal/f igural ability distinction 

Two reasons can be cited why ±;n this case the ^correlation 
between the factors is so high/. It will be recalled that 
the sample was drawn from a population of prospective 
elementary school teachers thus it is likely -to have C 
b^en all, or almost all, female and it has repeatedly been 
found that th^ f igural/spatial factors are weaker among 
females than among males (e.g.Werdelin, 1961). Secondly, 
it has been shown- that certain f igural .tests are quite 
amenable to solution with verbal/reasoning kinds of 
processing (cf. Gustafsson, 1976, ch 2). Most of the figural 
tests used by Behr are likely to be of that type; the tests 
on which verbal/reasoning strategies are less successful 
seem to be those with less complex stimulus configurations 
-HtiS^ which place higher demands on speed (such tests are 
classified as CFS in the SI system) . 

Furthermore it appears that the choice of tests and the 
choice of subjects interact to produce a high correlation 
between the factors , since females in particular appear to 
^p6sort to verbal/reasoning strategies v/henever possible 
(Gustafsson, 1976, ch. 6). 



In spite of the high correlation bdtyeen the faotolrs the 
2-f actor' model will be*used\as a^olrreasurement model for 
the aptitude variables^ However, the 1-f actor 'model will 
bemused as well, to contr^^.^'f^T^iterpretations' in terms^of 
general ability with interpreations in terms of^the 
verbal/figural ^distinction, , , ■ . 

■ . ■ ^ 

M^asurdment modelW^^for the depetndent variablQ^ 

In the measurement models for the dependent variati'les'^ ' 
only the sub- test scores (L^, LB, RA, afid RB) will bfe 
used; the total scores- can of course not be included 
since they are linearly dependent on the sub-test .scores , 
, and the TP measure cannot alone define a "time-factor." 

i 

With 4 dependent variables it is possible to define a 
2-f actor mode] and still have 1 degree of freedom left 
to test goodness of fit. It should be pd'Ssible/ however, ^ 
to define two potentially meaningful 2-f actor solutions: 
either one'^ learning factor defined by LA and LB and one 
retention factor . defined .by RA and RBj-or one computational 
speed factor defined by LA and RA and offe"^ factor defined 
as understanding of the structural properties of the system 
measured by LB and RB. Both these possibilities will be 
investigated. 

Basing the analysis on the correlation matrices the lear- 
ning/retention factors were first postulated. In both 
groups an exceedingly poor fit was obtained (VS: = 
49.9, df = 1; FS: x^' = 29.0). Testing the equality of 
the estimated parameters for the two group^, no significant 
difference was found (x^ = 6,0, df = 9) . 

The poor fit Within both treatment groups may in 

this case be due to correlated errors of measurement; 

the common content of 'the LA .and RA tests on the one hand 

and of the LB and RB tests on the other can be suspected 

'to cause 6 and 6 to be larger than zero. Since 

y LA,RA LB,RB 
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the model has only 1 degree of freedom it is not possrijDle 
to estimate these two correlatedi- errprp here; when 
variables , are' added this is, howevei:, pdssiible (Joreskog 
& Sorbom, 1978, pp, 22-28) • 

It. is nevertheless possible to obtain an indi^rect^ajipra-isal 
of the effects of the correlated ' e;rrors of measurement, A 
max'imum likelihood estimate 6f th^ correlation^ between 
Ht and J]- can e,asily be obtain^ (Joreskog & ' Sorbom^ 1977,^ 

p. 293) . If this estimate is lower than the estimate ob- 

\* • . 

?J:ained when the assumption is made that the errors ^UDf 
measur'ement kre uncorrelated, thrs can only pe because 
the correlated^errors contr^ibute to the latter estimate • 
Allowing for possibly correlated err6rs the estimates -^f 
tlie correlation betwedn r]^ and t]^ were .79 and .91 in 
theVS^and FS groups, respectively? a'&.suming uncorrelate4^:^ 
errors the corresponding estimates^er^ ^99 and 1.08. Thus, 
when^ uncorrelated errors of measurement are assumed t}ie 
learning and retention factors collaps.e, but when the 
effects of common test content are partialled out the 
factors appear to be distinct, at least within the VS 
group. 

Within the measurement model only it is not possible to 
make proper tests of whether one or both of the sub-tests 
contribute with correlated errors. VJithin the full LISREL 
model, where the aptitude factors have been added it is, 
however, possible to obtain t-tests of the estimates of 
these parameters. Bringing in the^e results already here 
it is found that the t-value for the estimate of the 



covardance . between the errors ^in LA and RA is 5.06 and 
2.92 in "the VS and FS groups, respectively? for LB and 
RB the corresponding figures are 1.65 and 1.05. Thus, 
the correlated errors are higher for computational speed, 
and particularly so in the VS group. 



Behr. found a significantly higher variance for RA arid 
.RB in the VS treatment , (Behr , 1967.^ pp. 42-43). It can 
thus be^fpected that differences are found between the 
treatment groups when the covariance matrices are analyzed 
instead of the correlation matrices . Th^jS" is indeed the 
case';'^there being, a highly s(iqjiif icajr^t value for the ffest 



statistic (x = 31.9, df = 9). pt w^ul^ of course also be 
possible to apply I>ISREL to test subsets of the parameters 
to get information about which variances differ between' ' 
'the treatments ; ^his is hardly necessavry, thoifgh/ since 
the conclusions from such ai^alyses are^'not likely to'^ 
differ from the- conclusions drawn by Behr. 

^. ■ • . ' V ■ 

.The correlated errors of measurement found above are 
clear^^ indications that LA and RA form one factqr and that 
LB and RB form another factor. Testing the hypothesis, of 
this 2-factor structure among the dependent variables a' 
somewhat better fit than when the learning/retention '\ 
factors were ^yestigated is obtained in both groups 
(VS: = 2lji0f, df = 1; FS: x^ = 4.1, df - 1, p < .04). 

The fit is obviously better in the FS group than in the, 
VS group? still a test of'^ythe ^'equality of the estimated 
parameters (f^^om the congelation matrices) is far from 
significan^t (x = 4.6, df = 9). Since it was shown above 
that the learning/retentiqp factors are identifiable, at 
least in the VS group, the reason for the poor fit of' 
the speed/understanding factor structure must be cox^- 
lated errors of measurement. 



AMowing for correlated errors of measurement the^stimates 
of the correlation between the factors were .65 and ^72, in 
the VS and FS groups, respectively. Assuming the errors to. 
be uncorrelated ' the estimates were .70 and .75. These 
figures indicate that the speed/under standitig' factors are 
stronger than are the learning/retention factors, and that 
the differentiation between the factors is more clear in 
\he VS than in the FS group. Using results from the full 
LISREL model the t-value for the covariance between the 
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errors in LA and LB is found to be 1.44 and 1.39 Hn the ^ 
VS and FS groups, re'Specti-yely ? for ,RA and RB th^.^'torres- v 
.ponding f igUres-^re arTg^ J^^ift ^t" ±^ foUnd that / • 

t^e correlated er/o^-s* are more important in. the group. 



The analyses of the dependent variables thus^'have siiown 
• that it is g)ossible ^ define two measuremefft models; ^one \ 
with the speed/under|;tanding factors, withr* tfie effects of 
^ -the' two occasions of . measurement ta]<^n into account? and ' 
.^^ne with the learning/retention factors, with the^^'ff ects ' 
of the common content - in tfie test taken inito account- \ 
Both these me^|urement models will be used. in the analysis . 
,of the structural relations within treatments. , - \ 
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Structural- r^lati ons .within treatments . " ' ■ r.^^^'-^' ^^ju'^^u'r/rii- 

Having developed the mieasureiinent models for the aptitude k ]'i4'::J-^ff-^\;: 
vari9ibles and for thb outcome variables it is now t^me to 
fit these together into ^the full LISREL model arid to study 



the structural^. relations within treatments. " 

r . . ■ ■ ■ y -.. N^^--* 

It wa^ decided tp i:^se,two different measurement models v^"^ 
; for the aptitude variables, otie; 1-f actor model only,-- „ ' * 
, representing^ a general fa^^tor* and one 2.-factor, model with ■ 
/the verbal/f igural factors. Also for the (dependent 'variables 
two measurement /mode'^ls were defined, one containg the 
learning/retentlpn factors 'and -one conta'inirig the speed/ 
understanding f^actors. Combining these measuremen^t models /. 
we get 4 full LISREL models. 

Ln Figure 1 the LISREfL model for the verbal/f ^^ural and ^ ^ 
speed/understanding factors i^ shown, using th% following, / 
symbols/: Latent variables are enclosed in circles observejci 
variables are enclosed in squares^ and errors of measure-, 
ment and disturbance 't-ferms are included without- being „ : 
^enclosed. A straight one-way arrow indicates a' causal . .-r " ^ 
influence of t!)ne variable gn anothe^s* and curved two-way ' ^ 
arrjdws indicate, correlation between variables without • any 
causal .interpretation (cf . Joreskog i& Sorbbm , 19718) . ' ' ' 




Insert Figure 1 about here 

' — 
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Only one figure- is shown but since there are two treatment 

K ^ . , ' • . " 

groups it' must be imagiried tha.-^ thbrfe is on^ graph' for each 

^ ' .. . ■ • J) . , \ ' 

treatment. The models for the two groups can have more or 

less in ^common, .however. One possibility is of course to 

estimate" all the parameters within the groups without 

#ons;training any of them to be equal: However in this 

case it Vis reasonable that the same ^measurement model for 

the aptitude' variables (SM) ^s used in both groups. Should 

the resuJLts firom. such an analysis differ from the results 

when different measurement models (DM) are used "^his can 

only be because' differences between the treatment groups 

^ with respect to the aptitude variables spuriously affect 

the relations between aptitude and. outcome within treat- , 

ments. Results obtained under both the DM and SM models 

will be presented. 

y 

The unstandardj^ed coefficients of the relations between - 
latent aptitude variables and the speed/understanding 
outcome variables are presented in Table 4, ,for both the 
1-factor and the 2-factQr aptitude models Along with 
the coefficients, t-values for pairwise test# of their 
similarity in the treatments have been -entered, whiile 
results from overall tests of fit are presented in' Table 5. 



Insert Tables 4 and 5 about here 



None of ttre^ests of interaction ^fcei^ approaches signifi- 
cance. However, a partial expla^nation of this is found, if 
the standard errors of the estimates are considered. These 
are particularly in the yerbal/f igural model, so high that 

only, few of the within-treatment relationships are signifi- 

•t 

cant. Since the coefficients presented are the unstandardized 
ones their absolute level cannot be judged from these figures 
alone. However, the standardized coefficients, interpretable 
as partial correlations, were quite high, with tha values 
often being around .50.' 



^ / - ' 

When the general aptitude variable is included in the mod^l 
the standard errors ^^nerally are smaller 'and the within- 
treatment coefficients are all highly significant. This 
indicates .chat the standard errprs are so large when the 
verbal/f igural variables are used because of the high 
correlation between the late^it aptitude variables, and 
from the formula for- the .standard errors of estimates of 
the Y coefficients it is clear that these are a quadratic - 
function of the correlation. ^ . 

'Looking at the descriptive pattern of results for the 
verbal/f igural aptitude variables it can be observed that ^ 
with respect to understanding the figura-1 aptitude \^riable 
has a higher coefficient in the ,VS group than in the FS 
group and tHat the verbal aptitude variable has a sliglftly 
higher coefficient in FS; with respect to speed the pattern 
of differences is completely revers'^d.^ It can also be 
observed that there are some differences in the descriptive 
patterns of results under the DM and SM models: With respect 
to understanding the SM model yields smaller differences 
between the treatments than does the DM model, while the 
opposite is the case for speed. 

For the general aptitude variable there is for both the 
outcome variables a higher coefficient in the VS than in 
the FS group, but it must also be observed that there 
especially with respect to understanding is a smaller 
difference between the treatments under the SM model than 
uhdei:. the DM model. 

No graph is shown for the model with the learning/retention 
outcome variables. This LISREL mpdel is quite similar to 
the one for speed/understanding, the only differences being 
that there is a unidirectional influence of learning on 
retention and that LA and RA now have correlated errors of 
measurement, as have LB and RB. 
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^In Tables 6 and 7 the results obtained with the learning/ 

retention factors are shown. The tests of interaction 
' yield even lower values of the test statistics than was 



Insert Tables' 6 '-and 7 about here 



the case for the oth^r ' putcome variables, and descriptively 
th'ere are for the verbal/f igural. variables only very small 
differences between tije coefficients for the treatments. 

For the general aptitude variable there is with respect 
to both, the outcome variables a somewhat higher coefficient 
in the VS group than in the FS group, but again the difference 
is smaller under the SM model than under the DM model. 

To summarize^, no significant interaction is found in the • . 

LISREL analyses even though there descriptively are some 
differences between the treatments: For computational speed 
a pattern of differences between treatments conforming to 
that specified in the original verbal/f igural ATI hypothesis 
is found, but for understanding the reverse pattern of 
differences is found. 

With respect to the learning/rentention factors there are 
even descriptively very few signs of interactions. It can 
be noted , however , that treatment obviously in some ways 
had differential effects on outcome: The . variance in the 
variables measuring retention was higher in the VS treatment 
than in the FS treatment and there is a lower correlation 
between learning and retention in the VS group than in the 
FS group. These differential effects may .6.f course be 
related to aptitude variables,, but if that is so it must 
be other variables than the verbal/f igural ones. 
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DISCUSSION AND CONCLUSIONS r 

Before discussing the methodological and substantive conclusions 
to be drawn from, this reanalysis it should be pointed out that 
a replication of the Behr (1967) study has been presented by 
Behr and Eastman (1975). In the replication a sample of about 
the same size was drawn .from the same population as in the 
original stucjy. Some changes were made of the instructional 
materials in that the figural treatment was made more indue- 
tive and the verbal treatment more deductive. In the replica- 
tion only two outcome measures were used, a retention and a 
transfer test. Both these tests were administrered one week 
after the learning session. Seven aptitude variables were 
'used? among those .were the tests that had shown interactions 
in the original study. 

In the Behr and Eastman (1975) analysis several different 
methods were used in the study of ATIs : Correlation coeffi- 
cients were compared, and a large number of multiple-regression 
analyses were performed. The dependent variables also were 
regressed on two Varimax factors. In no analysis any significant 
interaction was found and on the basis of these negative 
results Behr and Eastman warned against accepting the results 
from the original study. 

However, the replication did not assess computational speed 
and understanding as separate outcomes, and the reanalysis 
presented above indicates both that different patters of 
results are obtained with respect to these" outcome variables 
and that the- strongest effects are "found with respect to. 
computational speed. Therefore it does not seem that the, 
replication has much to tell about the results obtained in 
the reanalysis . 

As was mentioned earlier there exist at least three analyses 
and interpretations of the results in the Behr (1967) study. 
These will now be examined in the "light of. the results 
obtained using the LISREL approach. 



Behr himself ^tressed, in the Guilford (1967) tradition, 
the uniqueness of each single aptitude variable and 
refrained from relating these to each other. In contrast 
with the LISREL analysis Behr did find some significant 
interactions some more actually than can be expected 
from chance at the significance level chosen, if indepen- 
dence is assumed. The tests entering interactions with 
one or,.more of the post-tests w^re CMU, MM^l and CMC, 
which is a thoroughly mixed set, and it is hard to find 
any reason why these verbal tests and not the others 
should enter into interactions. 

4 

Using large sets of aptitude variables the ATI resfearcher 
i's exposed to very great risks of false postive and false 
negative conclusions. Thus with such an approach it is 
even in the long run almost impossible to sort out the 
dependable :^indings from the undependable ones and to 
know whether it is the unique or the common parts of the 
variables that enters into interactions. 

There is of course a risk that important interactions 
are missed if the specific parts of the aptitude variables 
are left 'out. However, if it cannot be shown on' theoretical 
grounds that it is the specific >arts of the aptitude 
variables which are likely to enter into interactions with 
the treatment variables there are strong reasons to reduce 
large sets of aptitude variables to a smaller set of 
latent variables. 

In order not to create any misunderstanding, we should 
point out that this recommendation of course only applies 
when it is reasonable to impose a factorial structure on 
the aptitude variables. If the tests all measure 
-different *f actors such an approach is not reasonable. 
However, even irj such cases LISREL can be used to *fetudy 
relations between latent variables,^ if each test is entered 
as two half-tests or if previously obtained estimates of 
the reliability -is entered into the model. • 



Cronbach and Snow concluded, as described in the 
Introduction, that a general factor of ability was more 
associatek with achievement r^)j.n* the VS treatmeflt than'i^ 
in the FS treatment. The LISREL results show that the 
coefficient for the structural relations between the 
general aptitude factor and the outcome variables tends 
to be considerably higher in VS than in FS.' However, a 
part of the interaction found by Cronbach and Snow can 
be accounted , for by differences between the treatment 
groups with respect.^ to the aptitude variables, since 
when the same measurement* model is used for both treat- 
ments the differehce between the coefficients is lower. 
With respect to con]putational spe,ed there is even- under ' 
the SM model a considerably higher coefficient in the 
VS group, but when the verbal/f igural aptitude variables 
are analyzed instead this difference is found to be . . 
wholly accounted for by the verbal ability variable. 
Thus, the LISREL results give very little support to 
the Cronbach and Snow conclusion. 

Gustafsson (1974) concluded that the interaction with 
verbal ability seemed well established in Behrs ;:Study 
and suggested also tha_^ there may be a very weak inter- 
action with figural ability. Gustafsson (1976)- also 
pointed out that -the tests of verbal ability -were not 
highly correlated with achievement in the VS treatment, 
but that they had no or a negative relationship with 
achievement in the FS treatment (cf. Behr & Eastman, 
1975, p. 156). I*t was concluded that this indicated 
that "pupils with a high verbal ability did poorly in 
the FS treatment rather than well in the VS treatment" 
(Gustafsson, 1976, p. ^'6). 

TKe- results for the understanding variable certainly do 
not give any support to such a conclusion but the results 
with respect to computational speed do to some extent: 
there is a negative coefficient for thfi relation between 
verbal ability and computational speed within the FS 
treatment, while at the same time there is a weak positive 
relationship within the VS treatment. 



To the best of our knowledge computational speed and 
similar types of outcome have not been much studied in 
ATI research studying the verbal/f igural aptitude variables, 
in spite of the fact that many studies have used mathematics 
as subject matter. It is thus difficult to compare this 
result in the Behr study with results, in other studies. 
However, the lack of interaction with respect to under- ^ 
standing is in agreement with the results in other studies / 
including the Behr and Eastman (1975) replication and 
studies by Bracht (1969); Carry (1967), Webb and Carry 
(1975), Hancock (1975), and Gustafsson (1976). 

No attempt will here be made to interpret the tendency 
towards interaction with respect to computational speed 
since it is so utterly weak. However, should further 
research provide supporting evidence , it seems . reasonable 
that the interpre'^ation Should be couched in what Gustafsson 
(1976, p. 81) labelled interferential terms, i.e. that a 
tre^atment has negative effects on the learning apd processing 
of subjects high in an ability. 

So far we have mainly discussed the results from a 
substantive point of view; now we will turn to a discussion 
about the methodological aspects. Before scrutinizing the 
advantages and disadvantages of LISREL as a method for 
analyzing ATI data, it must, however, be pointed out that 
suggestions for methods of analysis purporting to solve 
some of the problems which can be appi^oached with LISREL 
have been offered earlier. 

Cronbach and Snow (1977), p. 39) argued that when there - 
are several aptitude \ariables a reduced rank analysis 
may bo performed th.ough regressing t)utcome on aptitudes 
f ormeLi as composites , either on the basis of factor 
analysis or on the basis of judgement. And in the context 
of an illustrative analysis of a study including measure- 
ment of achievement at several points in time Cronbach and 
Snow (1977) claimed thuL methods such as path analysis 
and structural equation models, "are becoming increasingly 
signifierant in educational research and that they may be 
peculiarly valuable in .studie;s of learning and ATI" (p. 94). 



Some of the problems which may be solved with LISREL 
can certainly be solved with other methods. In order to 
reduce several observed variables to few latent variables 
component or factor analysis can be used? to study rela- 
tionships between true variables rather than observed 
variables correction por attenuation may be employed? 
and to study relations between measurements with an 
intrinsic causal ordering path analysis cah be used. 
But with LISREL it' is possible to specify models including 
all these features, which results in a more parsimonious 
and often more efficielit ana>3,ysis. Furthermore, LISREL 
offers capabilities not to be found in any other method 
^of analysis , such as the possibility of treating corre- 
lated errors of measurement, and to test goodness of fit . 

In the context of ATI research the LISREL approach also 
has some more , specif ic advantages. Even though it is 
seldom investigated, it is surprisingly often found that/ 
there are differences between the treatment groups with . 
-respect to the level and structure of the aptitude 
t^ariables (cf. Cronbach & Webb, 1975; Cronbach.& Snow, . 
1977, p. 38? Gustafsson, 1976, 1977, 1978) and such 
differences often result in spurious AT Is (and can for 
that matter also be suspected to conceal ATIs at times) . 
The necessity of formulating explicit measurement models 
for the aptitude variables in LISREL makes it, however, 
natural to investigate the similarity of the structure 
of the aptitude variables in f^he treatment groups. LISREL 
also allows investigation into' the effects of the 
differences between the treatment groups on the structural 
relations within treatments ^through comparing the results 
when the same measurement model is used and when ^^if f^^^^it 
measurement models are used, even though great caution is 
always necessary wften large and systematic differences are 
found between "the structure of the aptitude variables for 
the treatment groups. 

■ ' 90 ■ 
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So far we have only deal'lt with the possible ' advantages of 
LISREL as' a method for analyzing ATI data, but there are 
disadvantages as well. For one thing the^ method is built 
on rather strong assumptions: multivariate normality is 
assumed . The same assumption is found in regression 
analysis / but it is probably fair to say thctt any advantage 
that LISREL may have is reduced when this assumption ^is 

A'- 

not fulfilled. It must also be pointed out that the good- 
ness^of fit test is a large sample test. Unfortunately 
very little is known ^a^i)ut at what sample sizes 'the test 
sufficiently welj.. approximates its asymptotic properties, 
but obviously a warning must be made against using L'iSREL 
for very small samples. 

The greatest problem, however, in applying LI^il^L to 
analyze ATI data Is that the standard errors of the - ^ 
estimates of the structural relations within treatment^'^ 
are large. One reason why these standard errors j^re so 
large is that there tends to be an inverse r^ationship 
between the degree of consistency and the variance of 
an estimator, so there is a price to beypaid to obtain 
the unbiased estimates. (Parenthetically> it can be 
pointed out- that even within regression analysis the 
inverse relationship between consistency and variance 
has been studied, -see Winer, .1978) . Another reason why 
the standard errors are so extremely large in th.is 
particular case is that the correlation between the latent 
aptitude variables is so high. 

Even when regression analysis is used the power of the 
tests of interaction mostly is too low with the sample 
sizes which are feasible in ATI research (Cronbach & 
Snow, 1977) • If LISREL is to be used instead of regression 
analysis this problem is aggravated to become an obstacle 
against ever finding a significant interaction , at least 
within areas of study where there tend to be high correla- 
tions between 'the aptitude variables. 
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What can be done to reduce the large standard errors of 
the estimates? One solution is to increase sample size. 
However,' this solution may be^simple in theory but. need 
not be so in practice. Since ATI research is^ofteri 
experimental the data are cumbersome and costly to 
collect so it iftay simply be impossible to obtain the 
large samples needed. 

A more feasible strategy is to select the sample, the 
tests and the model so as to reduce the correlation 
between the latent variables. Reasons were stated above why with ' 
this sample and with these tests a large correlation 'can 
be expected with the verbal and thj^e figural aptitude 
variables, and in retrospect it cSn be concluded that 
neither were well chosen to study the verbal/f igu:^al 
ATI hypothesis. However, with the importance of having 
a low correlation between the latent variables in mind 
the researcher can often chose tests and subjects so as 
to minimize it. 

It is also possible to specify LISREL models which reduce 

the correlation between the aptitude factors. One 

/ 

possibility is to allow correlatedr^rrors of measurement 
/instead of invoking additional fact^ors. Another way to 
iow^T the correlations between the latent variables is 
to allow some of the observed variables to load in more 
than one factor. 

Using the' six tests in Behr's study with the highest 
communality it was in fact possible to define an orthogonal 
2-f actor model using such methods. However, the results 
from application of this measurement model have not been 
presented^ in full for two reasons. In the first place the 
factors were quite difficult to interpret; some of the 
verbal tests had, for Gxample, their highest loadings in 
the figural factor. SecG^idly, when this measurement model 
was used together with observed dependent variables (the 
measurement models used here for the dependent variables 
were not identified with this measurement model for the 
aptitude variables) the verbal factor disappeared in the 
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sense that it had a non-significant variance. 

These experiencl^s indicate that in this case the information 
in the observations ■ about the verbal/figural .aptitude variable 
distinction is too limited, whichever model is chosen to 
describe the data. 

A more drastic solution is of coursfe to specify an orthogonal 
model without bothering about the resulting poor fit when the 
variables are in fact correlated. To study the ^ffects of 
chosing an orthogonal model instead of the proper oblique 
model the computational speed/understanding outcome variables 
were analyzed together with the verbal/figural aptitude 
variables, defined by' the 12 tests but with a zero covariance 
•specified. 

For computational speed the difference between the within- 
treatment- coefficients for the VS and FS groups changed 
from 1.25 to . 98 for verbal ability and from -.12 to ^.01 
for figural ability. Thus, specification of the wrong model 
did not affect the estimates too seriously. However, the 
standard drrors were reduced with a factor of about 2.5 so 
in the orthogonal model there was in fact a significant 
difference between the within-treatment coefficients for 
the relation between computational speed and verbal ability 
(t = 2.08) . 

In spite of the fact that it is even possible to find a 
significant interaction in this case we/ certainly cannot 
recommend thatpoor -fitting orthogonal models are used. 
The estimates of the parameters need ^ot be too wrong, 
but the standard errors will be seriously underestimated, 
so the tests of interaction cannot be trusted. 

As was pointed out previously the problem of- the low 
.power of the statistical tests in ATI research is certainly 
not unique to LISREL? it is just more pronounced in that 
method of analysis than it is in regression analysis. 
Cronbach and Snow (19 77) concluded that the low power of 



the test of interaction in regression analysis makes it 
necessary to place lower weight on formal statistical tests 
and instead consider the descriptive results. Cronbach 

(j^75) even claimed that: "Tl)e time'^has come to e^prcise 

'-^'^the null hypothesis. We cannot afford to pour costly data 
down the drain whenever effects present in .the sample 'fail 
to ^each significance' . " (p. 124) . 

We strongly agree that less emphasis should be placed on 
statistical inference , and that greater importance should 
be attached to description of the effects in the sample 
(cf • Gustafsson, 1976) . 

LISREL does offer some advantages in a research strategy 
based on description. For one thing, the latent aptitude 
variables can be supposed to be more or less invariant 
over different studies, which is important when the results 
from different studies are brought together. Furthermore,, 
the description is based on consistent estimates of the 
parameters and it is generally very parsimonious. 

However, there are disadvantages as well. As was mentioned 
above the .specific parts o^f^Jbhe variables are not studied 
-i;\^ich of cours^ is serious when the possibility of local 
effects is entertained. It can also be claimed that the 
LISREL approach, which involves estimation of parameters 
in a'model, tends to be .quite remote from the observations 
themselves. But more important is the fact that LISREL in 
its present version (LISREL 4) does not allow hypotheses 
on means; consequently it ie not possible to investigate 
the- ordinality/disordinality of an interaction, let alone 
to describe the interaction effects as they appear in the 
sample. It should be pointed out, however, that LISREL can 
be complemented with analyses using the COFAMM program 
(Joreskog & S6rbom,l976b) which does allow estimation of 
the inliercept as loiiO as there are no constraints on the 
structural relation coe: f icients . 

1 
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But even so the description is based on rather constrained 
models, involving strong assumptions about tha nature of 
the interaction effects and it can be claimed that the 
description of interaction effects should be as unconstrai- 
ned as possible (cf. Gustafsson, 1976; Gustafsspn & Svensson, 
1978) . Thus it can be argued that even as simple methods as 
descriptions, of cell means for levels of ability within 
treatments for descriptive purposes do have some advantages 
over more "Constrained statistical models, either these are 
regression models or structural equation models. 

Therefore we tbink that LISREL analyses of ATI data often 
ought to be complemented with close descriptions of effects 
in the sample using as unconstrained a model for the descrip- 
tion as possible, even though the latter type of analysis ^ 
has not been possible in this reanalysis for lack of ravi^ 
data. 
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Figure 1 The full LISREL model for the verbal/f igural 

aptitude variables and the speed/understanding 
dependent variables. 




Table 1 The aptitude variables in the Behr study 



Test 



Sl-cell 



No of items 



Tijne limit 



Gestalt Canpletion Test 


CFU 


20 




6 


Figure Classification 


crc 


14 




3 


Figure Matrix 


CFR 


15 




7 


Paper Folding Test 


GFT 


20 




6 


Mop Memory 


MFU 


19 




10 


Ctoject Manory 


MFS 


30 




3 + 


Wide Range Vocabulary Test CMU 


24 




6 ' 


Word Classification 


CMC 


20 




4 


Verbal Analogies 




30 




9 


Manory for viord Meanings 


MMU 


30 




5 


Sentence Ccnpletion 


MVIR 


►35 




10 


Object-Number Test 


MSR 


15 




5 


Addition/Subtraction Test 


MSI 


120 




4 


Following Direction Test 


(INT) 


8 , 




• 5 



recall 



Table 2 Results 
factor 


from tests of 
solutions for 


fit 
the 


of the 1 
aptitude 


- and 2- 
variables 




No. of factors 










1 






2 


Difference 




df 




A 


df 


P ' 


df p 


Equality-'-^ 

y^vKzj- y j.<Ju^o J.OO • J. 


132 


.02 


158.4 


131 


.05 9.7 


1 .00 


No equality''"^ 

over groups ijy.o 


108 


.02 


128.0 


106 


.07 11.0 


1 .00 . 


DiffcTGince 29.1 


24 


.22 


30.4 


25 


.22 





1) The parameters are constrained (or not constrained) to be 
equal in the treatment groups . 




Table 3 The factor loadings (standardized) in the 

2-f actor solutions for the aptitude variabl 



Factor 

1 2 Communality 



Test VS FS VS FS VS FS 



CFU .41 .19 .17 .04 

CFC .60 .57 .36 .32 

CFR / .74 - .51 . . 55 . 26 . 

CFT .69 ^.52 .48 .27 

MFU .67. -19 .45 .04 

MFS .51 .20 .26 .04 

INT". . 50 . 55 . 25 .' .30 

CMU ..58 .48 .34 .23 

CMC ^.44 .62 I .19 .38 

CMR . .66 .72 /.4 4. .52 

MMU .42 .52 .18 .27 

^4MR ^ .39 .11 .15 .01 
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Table 4 Coefficients of structural relations between the 
latent aptitude variables and the speed/under- 
standing outcome variables within treatments 



Speed Under standii;ig 
VS i^S^ t vs FS . t\ 

DM: ' , . 

Verbal .45(.76) -.55(.74) .94 .74(.62) 1.60(/^) -.78 

Figural .29(.23) .29(.1§) -.02 .39(.20) .04(.19) 1.27 

General .43(.14) .19(.08) 1.49 .61(.15) .37(.10) 1.33 



SM: 

VerbaJ. .24(.85i^ -1.01(.75) 1.10 .63(.70) 1.20(.85) -.52 
Figural .32(.23) .44(.20) -.39 .37(.19) .14(.23) .77 

General .37(.ll) .19(.08) 1.32 .53(.ll) .45(.ll) . .51 

StancJard errors are shewn in parentheses, j 

The t-values refer to appraximate tests of equality of the 
within-treatinent coefficients. 

In the DM models the parameters in the measurement model for 
the aptitude variables are not constrained to be equal in the 
treatment groups, v^ile in the £M models they are. 
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Results from tests of fit the full LISREL 
models with the learning /retention outcome 
variables. ^ 



Verbal/figural 
aptitudes 

,,2 



df 



s equal 243. 



ips 



1.66 



80 . 



General 
aptitude 



95 196 .01 258.00 200 .00 



not 242.29 192. .01 255.79 198 .00 

^cy the groups 



2.21 2 .33 



Vr^^e qC^s equal 275.29 



IPS 



^-^^^^s not 

the groups 



274.25 



1.04 



221 .01 237.08 224 .00 
217 .01 286.12 222 .00 
4 -85 .96 2 .62 
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Table 6 Coefficients of structural relations between 
the latent aptitude vari*ables and the^ 
learning/retention outcome variables within 
treatments. 







Learning 
VS FS 


t 


Retention 
VS FS 


t 


DM: 








- 


• 






Verbal 


.50( 


.59) 


.46(.73) 


.04 


" .47(.48) 


.56(.47) 


-.12 


Flgural 


.39( 


.19) 


.24(.15) 


.49 


.07(.17) 


.,-.03 (.10) 


.5* 


General 


.55( 


.14) 


.33(.08) 


1.36 


.22 (.15) 


.10(.06) 


.67 


SM: 
















Verbal' 


.37( 


.69) 


-.19(.76) 


.55 


.46(.56) 


.51(.45) 


-.07 


Figural 


.39( 


.18) 


.42 (.20) 


-.07 


.06(.18) 


-.02(.14) 


.35 


General 


.48(, 


.10) 


.36 (.09) 


.89 


.18 (.13) 


.13 (.07) • 


• .34 



Standard errors are shcv/n in paroritlicscs. 

Tiie t-va.lues re for to apju*Qxijtiato tests of ecjuality of tlie 
witliin-Lxoatnient: coefficients. 

In the i3M models tlie parameters in tlie measurement model for 
tlie aptitude variables are not constrained to be equal in the 
treatment groups, while, in tiie StA models they are. 



V < 
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Tabiel 7 Results from tests of fit of the full LISREL 

models with the speed/understanding outcome 
V variables . 



p 



Verbal/f igural General 
aptitudes aptitude 

df p df - 

DM: 

Y-parameters equal 244.54 196 .01 258.83 200 .00 

in the groups 

Y-paranieters not 240.43 IM .01 255.79 198 .00 

equal in the groups 

Difference 4.11 4 .39 3.04 2 .22 



SM: 

Y-parameters equal 275.17 221 .01 287.99 224 .00 
in the groups 

Y-parameters not 272.01 217 .01 286.12 ' 222 .00 
equal in the groups 

Difference 3. 16 4 .53 .87 2 .65 



ERIC 



44 



R eports from the Institute dC Education, University of Gdtebortt 

1. Andersson, Dcncjt-Erik & Nilsson, S-G: An application of the 
. critical incident technique to the study of job and 
training requirements of shop mangers. -June 1962. 



2. Bullock, Donald, H: Research in programmed learning: .de- 

scription of and Rationale for A Program of Experimental 
Analysis. November 1965. 

3 . Harnqvist , Kjell: Social Factors and Educational Choice. 

1965. j 

4. Harnqvist, Kjell: Relative changes in intelligence from 13 

to 18. December 1967. 

5. Dahllof, Urban & Lundgren, Ulf'P: A project concerning' 

macromodels Eor the curriculum process. A short presenta- 
tion. Project COMPASS 12. April' 1969. 

6. Andcrsson, Bcngt-Erik: Project YG ^Youth in Goteborg) - A • 

presentation of background, design,, instruments and 
populations. Project UG 9. June 1969. 

7. Dahllof, Urban: Ability grouping, content validity and 

curriculum process analysis. Project COMPASS 13. June 196' 

8. Dahllof, Urban: Research on oral examinjDtion within the 

International Daccalaureats : Some general points of view 
and some inferences from the psychology of interviewing. 
Project COMPASS 20. January 1970. 

9. Dahllof, Urban: The materials and methods of implementation 

in the development of the curriculum. Outline of a model 
and some illustra tionf? from Sweden. Paper read at the 
, Conference. of Comparative Education Society in Europe, 
Prague,* June 1969. Project COMPASS 21. January 1970. 

10. Dahllof, urban & Lundgren, ulf P: Macro and micro approache; 

combined for curriculum analysis: A Swedish educational 
field project. Paper read at the annual meeting of the 
' Aiperican Educational Research Association in Minneapolis, 
March 2-6, 1970, Project COMPASS 23. April 1070. 

11. Andersson, Bengt-Erik: Actual and perceived attitudes among 

adolescents and adults toward each other. A ^±;udy of a 
group of Swedish teenagers and their pareni^t^fohe Stug- 
project 3. April 1971. ' 

12. Andersson, Bengt-Erik: Parental and peer influences and 

adolescent peer orientation. Project UG 12. April 1971. 

13. Andersson, Beng t-P:r.ik & Ekholfii, Mats: The generation gap. 

Frame of reference and design of the Stug-pro jec t . The 
Stug-projcct 4, May 1971, . 



l*- Andcrsson, Bongt-Llrik & Ukholm, Mats: Swedish Youth of To- 
day. Description of a Research Project About the Genera- 
tion GAp. Paper presented to the Interantional conference 
on "Socialization Processes in the Contemporary European 
Youth" in Bratislava June^4-17 June, 1971. The Stug- 
project June 1971. ;: 

15. Marton, Ference & Sandqvist, Gjertrud: Learning while typing. 

TIPS 3. September 1971. 

16. Dahllof, Utban, Lundgron, Ulf P, & Sioo, Margareta: Reform 

implementation studios as a basic for curriculum theory: 
Three Swedish approaches. Reprint from curriculum Theory 
Network 1, 1971: Monograph Supplement: Elements of Curri- 
culum Development, edited by F. Michael Connely with the 
assistance of John Herbert and Joei Weiss, Project COMPASS 
39. December 1971. 

17. Lundgrcn, Ulf P: An empirical study of the teaching process. 

Theory design and methodology. Project COMPASS 40. January . 
1972. 

18. Ilarnqvist, Kjell: Training and career structure of educational 

researchers. Reprint of paper read at Colloquium of Direc- 
tors of Educational Research Organizations, London 10-12 
November 1971, arranged by Council of Europe, March 1972, 

* 19. Harnqvist, Kjell: Canonical analyses of mental test profiles. 
. v* Pro ject '^MID ^ ...*April l97i'l '^ ^^ 

20. Harnqvist, Kjell & Bengtsson, Jai*l: Educational reforms and 

educational equality. Contribution to a Reader on Social 
Stratification edited by Scase. Aprxl 1972. 

21. Bradley, Gunilla: Women's interest in promotioli in relation 

to job satisfaction and home and school background. 
September 1972. 

22. Asberg, Rodney.: An attempt at a Pedagogical Evaluation of 

the Development Studies in Lund 1969-70: Housing, Building 
and Planning. May 1971. ^ 

23. Mar ton, Ference, Fransson, Anders, Jonsson, Barbro Klenell 

Xnn-Charlotte & Roos, Birgitta: Differential effects of 
stress-including instructions on anixiety. TIPS 7. 
October 1972. 

24. Lundgren, Ulf P: Educational Process Analysis. Two articles. 

Project COMPASS 48. November 1972. 

. 25. Lundgren, Ulf, P: Pedagogical frames and the teaching process. 
A report from an empirical curriculum project. Paper read 
at the AERA mooting in Now Orleans, 26 Feb - 1 March 1973. 
Project COMPASS 51. February -4 973 . 

Andrao, Annika: La te~bloomors and alternative courses of 
study at the s^enior level of the compulsory^omprehensive 
school and qualification for academic education at the 
upper secondary level. Project COMPASS 54 . February 1973. 



26. 



EKLC 



21. Lundgrcn, Ulf P : Pedagogical roles in the classroom, A 

contribution to Eggloson) 1*973 : Contemporary Research 
in the Sociology of Lilduca t ion 'London : Methusen, Project 
COMPASS 56, . » 

28 Andersson, Bengt-Erik: The Gieneration Gap - Imagination or 
♦Reality? Paper presenteted to th^' annual meeting of the 
\ International Society for the Study of Behavioral' deve- 

lopment • The Stug-pro jcct 21 . May 1973 • 

29. Andersson, Bengt-Erik: Pirdject YG.^ (Youth in GoteborgO - ' 
A Study of a group of Swedish adolescents. Project U€l3. 
June 1973. 

30. Dahllfof, Urban: Data? on curriculum and teaching process: 

Do' they make any difference to non-significant test 
differences - undr what conditions? Feb 27 1973. Project . 
COMPASS 60. August\1973V ' 

31. Andrae, Annika s, Dahll^iC , Ur^an : Process Analysis of Non- 

Graded Rural Schools . in Sweden, Outline of an Evaluation 
Project . Project "PANa 5. Fabruary, 1973. 

32. Lundgren, Ulf P: & Wallin, Erik: Some NOtes on Education 

as a Science . September 1973 .f, 

33. Norvell, Bo: The, Class Size ^and the Teaching Process. 

Part 1. October 1973. , * 

34. llarncjvist, Kjcll^ Individual differences in higher education 

Selection or Adaption? Plenary lecture at the Congress 

on Methodology of Research in Higher Education, Rotterdam, 

3-5 December r973. Prbjec.t MID 9. December 1973. 

35. New Trends in Evaluation . 'Report t^rom a conference on 

educational evaluation with an rntrodjiictory paper by 
professor Robert ^Stake, .University of Illinois. 
Januarj^ 1974. * . ' 

■ ' ' ', ' ^ 

36. Gustafsson, Jan-Eric: Verbal versus Figural Apt i tude-Trear€^ 

mcnt Interactions . Review the litterature and an 
empirical study. Project ^MID 10. March 1974 . 

37. Lindblad, Sycrker: Simulation and^ guidance. Experiences 

from planning and evaluation .teaching with simulation 
designed f.or grade 6 in Swedish compulsory schools. 
Project-grou0 UPLM 24. April 1974. 

38. Gustafsson, Jan-Eric: Implications of interactipns for 

the experimontaL research on teaching methods'. Project 
/MID 12. June 197 4;. ' 

39. Mairton, FcroncC: On rion^vcrbaLim learning: 1. Level of 

pr'ocessing a'ncl; level of outcome. December 1974 . 



40. Marton, Fe'rcncG: On non-verba Lim learning:ll. The 

erosion effect of a Laskpinduced learning aloritlim, 
December 1974. 

41. Marton, Ference & Dahlgren, Lars-Owe: On non-verbatim 

learning: iii. The outcome s^>ace of some. basic concepts 
in economics. December 1974 • 

42. Marton, Ference: On non-verbatim learning: VI. Some 

theoretical and methodological notes. December 1974. 

43. Lundgrep/ Ulf P: Analyse von Curr iculumprozessc^n der 

engere Ansatz : Curriculum-prozess als instruktionsge- 
scbehen. Research Report from the MAP-group 1975:1. 
January 1975. 

44. Marton, Ference, Dahlgren/ Lars-Owe, saijo, Roger & J 

Svensson, Lennart: The Gotcborg Project on Nonverbatim 
Learning. Four invited papera presented at the /American 
Educational Research A5;socia tion , 30 March - 3 April, 
Washington, DC. April 1975. 

45. Kilborn, Wicjgo & Lundgren, Ulf P: A Contribution to the 

Analysis of arithmetic Teaching and Learning. A 
paper presented at .the first DIMO-workShop, 31st ^of 
August - 5th of September, 1975, in Wasserlos, BDR. 
Research Report from the MAP-group 197 5 : 14 . December 
1975. 

46. Gustafsson, Jan-Eric: Inconsistencies in aptitude-treatment 

interactions as a function of producers in the studies 
and methods of analysis. Project MID 16. March 1976. 

47 . Gustafsson , Jan-Eric : Differential effects of imagery in- 

structions on pupils with different abilities. Project 
MID 17. April 1976. 

48. Ekliolm, Mats: Social development in school. Summary and 

excerpts. Project SOS 23. May 1976. 

49. Stangvik, Gunnar: Approaches to the analysis of learner- 

task interactions and some implications for the study 
of pedagogical processes. Project YP 7. January 1976. 



EKLC 



50. Andrae, Annika: Non-graded instruction in small rural 

lower secondary schools . A presentation of the PANG- 
project. Paper read at the INTERSKOLA conference in 
Sveg, July 1976. Project PAtJ^G 20. July 1976 . 

51. Patriksson, Goran: Attitudes toward Olympic games of 

Swedish adolescents. Paper presented at the interna- 
tional congress of physical activity sciences in 
Quebec City 11-16 July 1976. September 1976. 

52. Gustafsson, Jan-I::ric: A note on the importance of studing 

class effects in apti tudo- treatment interactions. Pro- 
ject MID 19. September 1976- 



4« 



Gustafsson, Jan-Llric: Spatial ability and the suppression 
of visualization by readincj. Project MID • September 1976, 

Dahlgren, Lars-Owe & Marton, Ference: Investigations - 
into the learning and teaching of basic concepts in 
economics - a research project ons hijgher education. 
Paper presented at the Conc/ress ^of the European Associ- 
ation for Research and ncvelopment in Highar Education, 
August 30 - September 3, 1976, Louvain la Neuve, 
Belgium, September 197 6, 

Marton, Ference: Skill as an aspect of knowledge. Some 
implications from research on students, conceptions of 
central phenomena i|i their subjects. Paper presented 
at the Second International Conference on Improving 
University Teaching^ July 13-16, 1976, Heidelberg, F.R, 
Germany, September 197 6 . 

Andrae Annika (Ed) j Nfon-graded instruction. Research 
orcjanization and de^igyn. Adnunis tra tion and daily 
teaching cxpcricnca|5 |in siuaJl rural lower secondary 
. schools, luxpcr j ence^ 'from the PANG-pro ject . Paper read 
at the INTICRSKOLA cf^riference in Sveg, July 197 6, Pro- 
ject PANG 23, 1\ugust 1976 . 

Sandgrcn, Bjorn & Asbqrg, Rodney: On cognition and social 
change, A report from a pilot study regarding the effect 
of schooling on cognitive growth. and attitudes towards 
social change in Pakistan , October 1976, 

Sandgren, Bjorn: Relation between cognition and social 
development. January 1977, 

Entwistle, Noel: Changing approaches to research into 
personality and |learning. February 1977 , 

■' f 

Harnqvist, Kjell: jpr'irnary Mental Abilities at Collective 
and Individual ijfevel, October 1977 , 

Harnqvist, Kjell: Enduring Effects of Schooling - A Neglec- 
ted Area in Educational Research, October 1977, 

Harnqvist, Kj.ell: A Note on .the Correlations between in- 
crements. Cumulated .^Attainment and a Predictor, October 
197 7 . : . • ^ 

Gustafsson, Jan-Eric : The Rasch Modcpl for Dichotomous 
Items: Theory , Applications and a Comput^er Program, 
December 1977, 

Harnqvist, Kjell &; Stahle, Gun: An Ecological Analysis of 
Test Score Changes over Time, December 1977, 

Dahlgren, Lacs Owe: Effects of University, Education on t^he 
Conception of Reality, Paper to be presented at the 
Fourth' Interna tiona 1. Conference on Improving University 
Teaching. July 26-29> 1978, Aachen, F, R. Germany , May 1911 

Marton, Ference: Describing conceptions of the -world around 
us. Sept 1978 . S ^ 



G7. GimiaC'AJion, tliiii-l.lL- ic: .. l.iiiclbLul, "^I'oi.r; Lon : , The r.'jsch iuodol 
L*()]" dichoLoiuoMj; .L Lcm:; : A scvl u L 1911 of the conci i Uional 
OG Lima Lion probloin foi Lonq Lcstr^ and some tlioucj hts 
• about i. Lcm scircon incj [>i:(jccdu ro:> . June 1978. 

68. Markluncl, Ulla: Smokintj in comul.sary schools. Interviews 

an»d improvj. sat ions . Pro jcjcL Behavioral science tobaco 
^ research. September 1978. 

69. Svenisison, Lonnart: Some notcV.v', on a me thodolocjical problem 

in the .study of tl)c re La t.i onshii) between thoucjht and .\ 
^ , lanfjuacje. - Doner i binij the thoucjht content in terms Of 
di f f ercn t cohceptionu of the same phenomenon, September 
1978. 

/? ^ • - 

7 0 . lUirncjvii; t / K jel 1 Si G'us taf sson , Jan-Eric : International 

references to Swedish educational research: A citation 
study. October 1978.' 

71. Lyhock, L.oif: A ^Research Al^proach' ..in to science education 

at CUjLebprfj. Paper: presented at the Joint ICMI/ICPK/CTS/ 
lJMl*;SCO/il.)M-conro i:enc(? nn ' Coopoi.a tion be twr:e!i Science 
Teachei's and »Ma thema t.i os 'l/cMcher:; ' , Septojnber 17-23 , - 
1978, 'Institut fiir Didakttk der Mathematik Bielef bid , 
FKG: October 1978. 

72. Lyhock, l.eif: Studies of nia thcma tics in science lesson in 

Cotcborcj . Paper presen ted' at the Joint ICMI/ICPE/CTS/ 
DNUSCO/lDM-conf erence on 'Cooperation between Scince ' 
Te.ichers and Ma thonia t i cr; 'i*c.;aclVer.s ' , September 17-23, 

^1 978, J.nst.itut fTir Didaktil: cier Mathematik, BielefuJd, 

'fug, October 1978. 

73. Gustaf sson , Jan-Br ic & Lindstrom, Berner: Describing 

and testing aptitude-trea tnent. interaction effects ^ 
with structural equation models: Reanalysis of a 
study by M.J. Behr . December 1978. 



50 



